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ABSTRACT 



A three-dimensional space may be rendered in a com- 
puter graphics system by representing the three-dimen- 
sional space as a plurality of convex polyhedra and 
representing the containment relationships among the 
plurality of convex polyhedra. Connection relationships 
among the convex polyhedra, including the location of 
holes in the connected surfaces, are also preferably 
represented. Containment and connection relationships 
are preferably represented using a containment logical 
tree and a connection logical graph. Three-dimensional 
graphics representations can thereby be rendered in a 
real time using a low cost, graphics processor. Efficient 
movement through three-dimensional space along all 
six degrees of freedom is also provided by using a two- 
dimensional pointing device, preferably a mouse, in 
combination with a function selector, preferably a pair 
of keys on the mouse. Use of the mouse without the 
function selection keys allows movement in a first plane 
in the three-dimensional space rendition, and use of the 
mouse in combination with the first function selection 
key allows movement in a second plane in the three-di- 
mensional space rendition. Use of the mouse in combi- 
nation with the second function key allows rotation in 
three-dimensional space rendition. Preferably, the six 
degrees of movement are implemented using predeter- 
mined combinations of mouse movement and function 
key selection to provide user intuitive movement 
through the three-dimensional space rendition along all 
six degrees of freedom. 

45 Claims, 24 Drawing Sheets 
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COMPUTERIZED METHOD AND APPARATUS 
USING CONTAINMENT RELATIONSHIPS TO 

REPRESENT OBJECTS IN A THREE- 
DIMENSIONAL SPACE, AND FOR MOVING 5 
THERETHROUGH 

FIELD OF THE INVENTION 

This invention relates to computer-based graphics 
processing and more particularly to three-dimensional, jq 
computer-based graphics processing. 

BACKGROUND OF THE INVENTION 

High performance computers and work stations are 
used extensively for three-dimensional graphics pro- 
cessing. In three-dimensional graphics processing, one 
or more three-dimensional images is displayed on a 
two-dimensional graphics display such as a high resolu- 
tion graphics screen. Computer Aided Design (CAD) 
systems allow users to create a three-dimensional repre- 
sentations using lines, points and vectors. Three-dimen- 
sional image data may be used in the drafting and docu- 
mentation phases of product design, and may also be 
used to control product manufacture. 

Product designers, such as architects and engineers, 
still begin the design process on paper and communicate 
design concepts on paper. Although CAD has suc- 
ceeded at automating drafting, documentation and man- 
ufacturing it has met with limited success at "visualiza- 
tion". 

Visualization is the first step in creating a spatial de- 
sign. Visualization allows a user to create, explore, 
change, interact with and share a complete simulation 
of a three-dimensional world. It would be highly desir- 
able for a designer to use the computer to create, mod- 
ify and observe a three-dimensional space. 

Unfortunately, Computer-Aided Visualization 
(CAV) has not been heretofore used extensively be- 
cause of two main problems. First, the creation and 
manipulation of objects in three-dimensional space on a 
computer typically requires high performance comput- 
ers or workstations to produce real-time visualization. 
Second, efficient techniques have heretofore not been 
available for moving an observer through a three-di- 
mensional space on a graphics display along all six de- 
grees of freedom. Each of these problems will be de- 
scribed in detail 

The creation of a three-dimensional space on a two- 
dimensional display is often referred to as "rendering*'. 
It is difficult to perform rendering in real time using 
conventional graphics techniques. A major problem in 
real time rendering is detennining spatial priority. That 
is, it is difficult to calculate which objects are in front of 
other objects from a particular point of view or refer- 
ence position. 

A number of techniques have been developed to 
determine spatial priority. Many of these techniques are 
described in a textbook entitled Computer Graphics, 
Principles and Practice, by. Foley et al. (Second Edition, 
1990) published by Addison-Wesley Publishing Com- 
pany, Inc., the disclosure of which is hereby incorpo- 
rated herein by reference. However, these techniques 
have been so resource intensive that real time rendering 
typically cannot be performed without the use of expen- 
sive supercomputers or workstations. 

One technique which has been used to render a three- 
dimensional space is referred to as a "z-buffer". See the 
Foley et al. text, pages 668-672. In a z-buffer, a separate 
buffer is provided which includes a depth field for each 
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pixel on the two-dimensional display. When a three-di- 
mensional space is rendered on the display, a new value 
in the z-buffer is calculated for every pixel. The newly 
calculated depth is then displayed on a graphics display 
using well known techniques. Since repositioning of the 
three-dimensional data requires a recalculation of the 
depth value for each pixel, this technique is very re- 
source intensive. 

A second technique for recreating a three-dimen- 
sional space in a graphics computer system is referred to 
as "ray tracing" or "ray casting". See the Foley et al. 
textbook, pages 701-715. In this technique, the visibility 
of surfaces is determined by tracing imaginary rays of 
light from the viewer's eye to the objects in the scene. 
Since large numbers of rays must be traced in order to 
produce a high quality image of spatial priority, this 
technique also is typically not a real time technique 
unless high performance computers are used. 

Binary space partitions have also been used for three- 
dimensional rendering- See the Foley et al. textbook, 
pages 701-715. Binary space partitioning breaks the 
three-dimensioned space into a plurality of smaller 
spaces and each smaller space is rendered individually. 
The rendered smaller spaces are then combined. Again, 
this technique is resource intensive. 

Yet another technique for three-dimensional render- 
ing is the "painters algorithm" also referred to as a 
"depth-sort" algorithm. See the Foley et al. text, pages 
673-675. In the painters algorithm, the farthest objects 
are first rendered and then closer objects are rendered 
on top of the farther objects, with the closer objects 
masking the farther objects. Since this requires all of the 
objects in the three-dimensional space to be redrawn 
every time a viewing position is changed, the algorithm 
is time consuming. 

A final technique used for three-dimensional render- 
ing is "scan line** technique. See the Foley et al. text 
pages 885-887. The three-dimensional space is scanned 
pixel by pixel and rendered pixel by pixel. Since the 
rendering is performed individually for each pixel, this 
technique is highly resource intensive. 

In summary, the rendering of a three-dimensional 
space from a particular point of view is extremely time 
consuming because spatial priority determination is 
resource intensive. Accordingly, real time rendering 
has been difficult to perform on conventional desktop 
or personal computers. 

The second major "problem in real time three-dimen- 
sional computer aided visualization is allowing an ob- 
server or reference point to easily move in the three-di- 
mensional computer-generated scene along all six de- 
grees of freedom. As is well known to those having skill 
in the art, an observer of a real three-dimensional space 
can move in six degrees of freedom by walking through 
the space and turning his head. The six degrees of free- 
dom are translation along three orthogonal axis (often 
referred to as x, y and z axis) and rotation about the 
three orthogonal axis (often referred to as roll, pitch, 
and yaw). 

Unfortunately, typical computer input devices which 
control movement through computer generated three- 
dimensional scenes have been either a keyboard or a 
joystick. The keyboard is an unnatural movement input 
device because it requires multiple keystrokes to move 
forward, or it assumes a velocity set by numeric keys 
and uses other keys to change directions. Joysticks are 
an improvement over the keyboard but are useful pri- 
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manly for airplane simulators where user control about 
six degrees of freedom is not required. The joystick 
typically assumes a forward velocity, and the viewer 
uses the joystick to modify direction. There have also 
been a number of mouse based control systems used S 
with computer based flight simulators, but these have 
typically been based upon the control system used by 
the joystick. 

One attempt to improve the freedom of movement in 
a graphics display of a three-dimensional space has been 10 
implemented in a computer adventure game entitled 
**The Colony" which is described in a manual entitled 
"The Colony" published in 1988 by Mindscape, Inc. In 
this computer game, a cartesian grid was overlaid on 
top of the computer generated image, where the center 15 
of the grid is the center of the image. When the mouse 
is at this exact center, no motion occurs. As the. mouse 
is moved up along the vertical axis from the center, the 
observer's position begins to move forward through the 
three-dimensional space. The distance from the center 20 
determines the speed of forward movement. Return the 
mouse to the center causes all motion to cease. If the 
user moves the mouse vertically downward from the 
center, the observer's position begins to move back- 
wards. Again, the distance from the center determines 25 
velocity. The user can rum by moving the mouse along 
the horizontal axis. Movement to the left makes the 
observer turn left The angular velocity of the turn is 
determined by the distance from the center. Movement 
to the right makes the observer turn to the right, with 30 
the angular velocity being determined by the distance 
from the center. 

The above described movement input technique al- 
lows the reference position or observer to stop and walk 
backward. However, this technique does not allow the 35 
observer to efficiently move along all six degrees of 
freedom. In particular, it does not allow the user to 
easily move directly to the left or right, to change alti- 
tude, to change pitch or to change roll. 

In summary, there is a great need in three-dimen- 40 
sional graphics processing, for real time three-dimen- 
sional rendering -which allows a user to move through 
the three-dimensional space in real time along all six 
degrees of freedom, and which can be implemented 
using low cost personal computers. 45 

SUMMARY OF THE INVENTION 

It is therefore an object of the invention to provide an 
improved computerized method and apparatus for rep- 
resenting a three-dimensional space. 50 

It is another object of the present invention to pro- 
vide an improved method and apparatus for rendering a 
three-dimensional space on a two-dimensional graphics 
display in real time. 

It is yet another object of the present invention to 55 
allow a user to efficiently move through a three-dimen- 
sional space rendering along all six degrees of freedom. 

It is yet another object of the present invention to 
provide three-dimensional rendering and six degrees of 
freedom movement, without the need for supercomput- 60 
ers or specially designed high performance graphics 
workstations. 

These and other objects are provided in a computer 
graphics method and apparatus which represents a 
three-dimensional space as a plurality of convex polyhe- 65 
dra. A convex polyhedron is a three-dimensional spatial 
structure which is completely bounded by a plurality of 
flat surfaces, such that a line between any two points 
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inside the polyhedron does not cross any of the surfaces 
which define the polyhedron. Examples of convex 
polyhedra are a cube and a pyramid. 

According to the invention, since each convex poly- 
hedron includes an interior and an exterior, a polyhe- 
dron can contain one or more polyhedra and can be 
contained in another polyhedron. Accordingly, the 
computer-based representation of the present invention 
also includes means for indicating containment relation- 
ships among the plurality of convex polyhedra. 

By modeling objects in a three-dimensional space 
using convex polyhedra having containment relation- 
ships among one another, three-dimensional rendering 
may be performed in real time without high perfor- 
mance computers. Real time rendering may be per- 
formed because the calculations for rendering from a 
predetermined reference point are greatly simplified. In 
particular, if the reference point is outside a particular 
polyhedron, all polyhedra contained within that poly- 
hedron are not visible from outside it, and need not be 
rendered in recreating the three-dimensional space. 
Similarly, if the reference point is completely contained 
within a polyhedron, polyhedra outside this polyhedron 
also cannot be seen. Accordingly, these polyhedra need 
not be considered in producing the rendering. 

The present invention only requires calculation of the 
position of those polyhedra which can be seen from the 
reference position, based upon the containment rela- 
tionships. The entire three-dimensional space need not 
be rendered each time the reference position moves. 
High speed, real time rendering is thereby provided. 

It will be understood by those having skill in the art 
that objects contained within a polyhedron, or in which 
the polyhedron are contained, may need to be at least 
partially rendered if the polyhedron includes a hole in 
one or more of the surfaces thereof. According to the 
invention, the relevant images of polyhedra may be 
rendered through holes by also representing connection 
relationships among the surfaces of each of the plurality 
of convex polyhedra. Holes in the connected surfaces 
are also indicated. By allowing holes between logically 
connected surfaces of two polyhedra, a rendering be- 
tween the interiors of polyhedra may be performed, 
without having to conduct lengthy search operations to 
determine if the polyhedra are visible, and to find other 
polyhedra which are not in fact visible but whose im- 
ages overlap the hole. 

In a preferred embodiment of the present invention, 
two logical structures are maintained to define the rela- 
tionships among the convex polyhedra of the three-di- 
mensional space. The tree structure is a tree indicating 
the containment relationships among the convex poly- 
hedra. This logical structure for indicating containment 
will be referred to as a "containment tree". The second 
logical structure is a bidirectional graph which specifies 
the connections among polyhedra surfaces and also 
indicates where the connections include a hole. This 
allows the Tenderer to quickly determine what is on the 
other side of a hole in a polyhedron. This logical struc- 
ture will be referred to as a "connection graph**. These 
two structures are used by the graphics Tenderer to 
create a computer-generated three-dimensional graphic 
image in real time. 

The present invention also allows efficient movement 
through the three-dimensional space along all six de- 
grees of freedom. A two-dimensional pointing means, 
such as a mouse, trackball, joystick or other two dimen- 
sional pointing means, is used, in combination with a 
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function selection means. The combination allows the be used with other techniques for moving through a 

two-dimensional pointing means to move the reference computer-generated three-dimensional space rendition, 

in a second plane in the three-dimensional space in re- Moreover, the technique for moving through a comput- 

sponse to selection of the function selection means and er-generated three-dimensional space representation 

movement of the two-dimensional pointing means. In 5 may be used in conjunction with other computer-based 

other words, use of the two-dimensional pointing means representations of three-dimensional space. However, 

without the function selection means allows movement the representation of a three-dimensional space and the 

in a first plane in the three-dimensional space, whereas technique for moving through a three-dimensional 

movement of the two-dimensional pointing means when space representation, when used in combination, allows 

the function selection means has been selected allows 10 a real time computer aided visualization system to be 

movement in a second plane in the three-dimensional produced which can operate on relatively low perfor- 

space rendition. Another function selection means may mance computers. Efficient computer aided visualiza- 

also be provided to allow rotation in the three-dimen- tion is thereby provided. 

sional space rendition when the two-dimensional point- The present invention has been embodied in a com- 
ing means and the second function selection means are IS puter program entitled Virtus WalkThrough TM , and 
selected. This allows six degrees of movement using the has been marketed by the assignee of the present inven- 
two-dimensional pointing means. tion Virtus Corporation, Cary, N.C, since Nov. 15, 

In a preferred embodiment of the present invention, 1990. The computer program instructions for Virtus 

the two-dimensional pointing means is a mouse and the WalkThrough TM arc hereby incorporated herein by 

two function selecting means are two keys on the mouse 20 reference, as is the user manual for Virtus Walk- 

or two keys on the keyboard. Through tm entitled "Virtus Walk Through User's 

According to another aspect of the invention, user Manual". The present invention was first publicly dera- 

inputs from the two-dimensional pointing means and the onstrated by the assignee of the present invention at the 

first and second function selection means may be pro- Architecture, Engineering and Construction (A/E/C) 

cessed, as described below, to provide efficient user 25 Systems *90 Trade Show in Atlanta, Ga. beginning on 

movement in a three-dimensional space along all six Jun. 12, 1990. 

2?S 222S2Tttt^^ BRIEF DESCRIPTION OF THE DRAWINGS 

vertical movement of the two-dimensional pointing FIG. 1 illustrates a computer system on which the 

means translates the reference forward/backwards 30 present invention may be practiced, 

through the three-dimensional space rendition, and FIG. 2 is a graphical illustration of a three-dimen- 

horizontal movement of the two-dimensional pointing sional space, according to the present invention, 

means rotates the reference point to change the yaw. In FIG. 3 illustrates a containment tree structure for 

other words, using the mouse without the function keys graphically representing FIG. 2, according to the pres- 

allows the viewer to "move forward**, "move back- 35 ent invention. 

wards", "turn left", and "turn right". FIG. 4 illustrates a graphical representation of a plu- 

Vertical movement of the pointing means when the rality of three-dimensional rooms, according to the 

first function selection key is selected, moves the viewer present invention. 

up/down through the three-dimensional space axis, and FIG. 5 illustrates a bidirectional connection graph 

horizontal movement of the pointing means when the 40 structure for the representation of FIG. 4, according to 

first function selection key is selected translates the the present invention. 

reference left/right. In other words, use of the mouse FIG. 6 illustrates a bidirectional connection graph 

and the first selection key allows the reference point to structure for the representation of FIG. 4, including 

"rise up", "sink down", "pan left" or "pan right". holes, according to the present invention. 

Finally, when the second function key is selected, and 45 FIG. 7 illustrates a step in rendering the representa- 

the mouse is moved vertically, rotation is provided to tion of FIG. 4, according to the present invention, 

change the pitch. When the mouse is moved horizon- FIG. 8 illustrates the rendering corresponding to 

tally and the second function key is selected, rotation is FIG. 7, according to the present invention, 

provided to change the roll. In other words, use of the FIG. 9 illustrates a complete rendering of the repre- 

mouse and the second function key provides a "look 50 sentationof FIG. 4, according to the present invention, 

up", "look down", "roll left" and "roll right" function, FIG. 10 illustrates of the action of the pointing device 

Accordingly, the use of the mouse in combination with when neither function key is selected, according to the 

two selection keys allows movement along six degrees present invention. 

of freedom in a user-intuitive manner. FIG. 11 illustrates the operation of the pointing de- 

The computer-based representation of a three-dimen- 55 vice when a first function key is selected, according to 

sional space including means for indicating containment the present invention. 

relationships among a plurality of convex polyhedra, FIG. 12 illustrates the operation of the pointing de- 

and means for indicating connection relationships vice when a second function key is selected, according 

among the surfaces of the plurality of convex polyhe- to the present invention. 

dra, may be used to efficiently represent a three-dimen- 60 FIGS. 13A and 13B illustrate a coordinate system 

sional space on a two-dimensional display. The method used a three-dimensional space, and a representation of 

and apparatus for moving through a computer- a grid and cursor on a two-dimensional graphics dis- 

generated three-dimensional space including the func- play, respectively, according to the present invention, 

tion selection means and the two-dimension al pointing FIGS. 14-19 illustrate the operational steps for imple- 

means, allows movement along six degrees of freedom 65 men ting six degree of freedom movement, according to 

through the three-dimensional space rendition in an the present invention. 

intuitive and user friendly manner. The three-dimen- FIGS. 20A-20J graphically illustrate the results of 

sional space representation of the present invention may performing the steps of FIG. 14. 
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FIGS. 21A-21J graphically illustrate the result of 
performing the steps of FIG. 14, at a faster velocity than 
that shown in FIGS. 20A-20J, according to the present 
invention. 

FIGS. 22A-22E graphically illustrate moving back- 5 
wards upon performing the steps of FIG. 14, according 
to the present invention. 

FIGS. 23A-23E illustrate the combination of moving 
forward and turning right in response to the steps of 
FIGS. 14 and 15 according to the present invention. 10 

FIGS. 24A-24E graphically illustrate the results of 
performing the steps of FIG. 15 when turning to the 
left, according to the present invention. 

FIGS. 25A-25E graphically illustrate the results of 
performing the steps of FIG. 15 when turning to the 15 
right, according to the present invention. 

FIGS. 26A-26E graphically illustrate the results of 
moving up in response to performing the steps of FIG. 
16, according to the present invention. 

FIGS. 27A-27E graphically illustrate the results of 20 
moving down in response to performing the steps of 
FIG. 16, according to the present invention. 

FIGS. 28A-28E graphically illustrate the results of 
performing the steps of FIG. 17 for moving left, accord- 
ing to the present invention. 25 

FIGS. 29A-298E graphically illustrate the results of 
performing the steps of FIG. 17 for moving right, ac- 
cording to the present invention. 

FIGS. 30A-30E graphically illustrate the results of 
performing the steps of FIG. 18 for increasing pitch, 30 
according to the present invention. 

FIGS. 31A-31E graphically illustrate the results of 
performing the steps of FIG. 18 for decreasing pitch, 
according to the present invention. 

FIGS. 32A-32E graphically illustrate the results of 35 
performing the steps of FIG. 19 for rolling to the left, 
according to the present invention. 

FIGS. 33A-33E graphically illustrate the results of 
performing the steps of FIG. 19 for rolling to the right, 
according to the present invention. 40 

DESCRIPTION OF A PREFERRED 
EMBODIMENT 

The present invention now will be described more 
fully hereinafter with reference to the accompanying 45 
drawings, in which a preferred embodiment of the in- 
vention is shown. This invention may, however, be 
embodied in many different forms and should not be 
construed as limited to the embodiment set forth herein; 
rather, this, embodiment is provided so that this disclo- 50 
sure will be thorough and complete, and will fully con- 
vey the scope of the invention to those skilled in the art. 
Like numbers refer to like elements throughout. 

The preferred embodiment of the present invention 
may be implemented on a computer program which 55 
runs on a standard personal computer such as an App- 
le® Macintosh® system. This system will only be 
briefly described because it is well known to those hav- 
ing skill in the art Similarly, the present invention uti- 
lizes, in part, many standard graphics techniques for 60 
defining polyhedra and for generating a graphics dis- 
play. These techniques are described in detail in the 
Foley et al. textbook mentioned above, and will not be 
described in detail herein. Rather, the present descrip- 
tion will concentrate on the containment and connec- 65 
tion means and steps of the present invention. 

Similarly, hardware and software techniques for 
using pointing devices to control the position of a cur- 
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sor or other reference point on a computer screen are 
well known and will not be described in detail here. 
Rather, the description will concentrate on the tech- 
nique for using a pointing device in conjunction with 
two selection means to provide six degrees of freedom 
in moving through a representation of the three-dimen- 
sional space in a user friendly and user intuitive manner. 

The present invention may be implemented effi- 
ciently in a personal computer using object oriented 
programming techniques. Object oriented program- 
ming is well known to those having skill in the art and 
will only be described briefly herein. A complete de- 
scription of an object oriented computer program 
which embodies the present invention will be described, 
in detail. Specific examples will also be used to describe 
the invention, where needed. 

System Environment 

The present invention was designed to efficiently 
render a three-dimensional space and allow observer 
movement therethrough in real time using a low cost 
personal computer. Although the present invention 
may also be used to enhance three-dimensional render- 
ing and simulated movement in a high speed worksta- 
tion, mainframe computer, supercomputer, or any 
graphics system, it was first designed to provide these 
real time functions in the personal computer environ- 
ment. 

The invention may be used on any Apple ® Macin- 
tosh® computer that has at least two megabytes of 
memory. Although it is preferred to use a color display, 
a black and white display may also be used. The inven- 
tion may also be used on any other personal computer 
having a two-dimensional pointing device such as a 
mouse, joystick, trackball or keyboard. 

Referring to FIG. 1, the computer system 10 includes 
a display 11, preferably a high resolution color graphics 
display, and a central processing unit 12. The central 
processing unit contains a microprocessor (not shown) 
and random access memory (not shown) for storing 
programs therein for processing by the microprocessor. 
A disk drive 13 for loading programs may also be pro- 
vided. A keyboard 14 having a plurality of keys thereon 
is connected to the centra] processing unit 12, and a 
pointing device such as a mouse 15 is also connected to 
the central processing unit 12. As shown, mouse 15 
includes a pair of selection keys 16a, 16b which may be 
used in conjunction with moving an observer through a 
simulated three-dimensional space, as will be described 
below. It will be understood by those having skill in the 
art that many forms of pointing devices are used in data 
processing systems. Henceforth, any reference to mouse 
15 will be understood to refer to any pointing device. 

Three-Dimensional Space Representation-General 
Overview 

A general overview of the computer-based represen- 
tation of three-dimensional space according to the pres- 
ent invention will now be provided. The three-dimen- 
sional space representation represents all three-dimen- 
sional objects as combinations of convex polyhedra on a 
computer graphics system. As is well known to those 
having skill in the art, a convex polyhedron is a three-di- 
mensional structure which is completely bounded by a 
plurality of flat surfaces, such that any two points inside 
the polyhedra may be connected by a line which does 
not cross any of the surfaces of the polyhedron. Typical 
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examples of convex polyhedra are a cube, cylinder and The containment and connection relationships de- 
pyramid, scribed above may be maintained in a three-dimensional 
Once a plurality of three-dimensional polyhedra are graphics processing system by maintaining two logical 
defined, the connections therebetween are also sped- structures. The first logical structure is a tree structure 
fled. A connection is a logical construct which relates a 5 where the nodes are all polyhedra and the branches of 
surface on one polyhedron to the surface of another the tree are those polyhedra which are contained by a 
polyhedron. In one implementation, connection may node polyhedron. As is well known to those having skill 
only be maintained among polyhedra surfaces that lie in in the art, a tree structure may be defined as a hierarchi- 
a single plane. However, the planes do not have to be in cal structure consisting of nodes connected by 
the same physical space, with overlapping and congru- 10 branches, in which each branch connects one node to a 
ent surfaces. Since the connections are logical connec- directly subsidiary node; there is a unique node called 
tions, any polyhedron can be connected to any polyhe- the root which is not subsidiary to any other node; and 
dron regardless of their physical connection to one every node besides the root is directly subsidiary to 
another. The renderer, upon finding the logical connec- exactly one other node. See the IEEE Standard Dictio- 
tion, will start rendering the model based on the con- 15 nary of Electrical and Electronics Terms, Fourth Edi- 
nection. This property allows portals between volumes tion, 1988. An example of a three-dimensional space and 
that do not necessarily map into the same space. For the containment tree therefor is provided in FIGS. 2 
example, several large rooms that would normally con- and 3, respectively. 

flict in space could be connected to a surface of a small As shown in FIG. 2, a plurality of polyhedra PI, P2, 

lobby to allow easier, quicker navigation between the 20 P3, Ql, Q2 and R are all contained within a particular 

large spaces. world polyhedron W. The world polyhedron W is the 

The three-dimensional space also includes a "refer- container of every polyhedron in a three-dimensional 

ence" which is also referred to as an "observer" or space. In other words, all polyhedra contained within 

"viewer" and which is defined as the point of view or the three-dimensional space are contained within the 

camera position from which the image will be rendered. 25 world polyhedron. As shown in FIG. 2, polyhedron R 

As will be described below, the present invention al- is contained within polyhedron PI, polyhedra Ql and 

lows the reference to be moved along six degrees of Q2 are contained within polyhedron P2 and polyhedra 

freedom using a two-dimensional pointing device and PI, P2, and P3 are contained within the world W. 

two selection keys. The containment tree structure for the three-dimen- 

According to the invention, the "containment" rela- 30 sional space of FIG. 2 is shown in FIG. 3. This tree 

tionships among the plurality of convex polyhedra in structure logically describes the relationship described 

the three-dimensional space are used to efficiently pro- above in words. By using the containment tree, the 

duce three-dimensional space rendering in real time. rendering operation is greatly simplified. For example, 

Since all of the three-dimensional shapes have been if the observer is inside PI, the only possible polyhedron 

defined as convex polyhedra, the present invention 35 which needs be rendered is R, and P2, Q2 and P3 may 

recognizes that convex polyhedra have both an interior all be disregarded. Similarly, if the observer is in P2, 

and an exterior. Therefore, polyhedra can contain other only Ql and Q2 need be rendered and PI, R and P3 may 

polyhedra. If the containing polyhedron is completely be disregarded. High speed, real time rendering is 

opaque, the contents of the polyhedron are not visible thereby provided. 

from a reference outside the polyhedron. Accordingly, 40 The second logical structure is a bidirectional graph 
those internal objects to the polyhedron do not have to which specifies the connection relationships among the 
be considered when attempting to draw a scene from a polyhedron surfaces. As is well known to those having 
position outside the polyhedron. Conversely, if the skill in the art, a graph may be defined as a model con- 
observer is contained within an opaque polyhedron, sisting of a finite set of nodes having connections called 
there is no need to consider objects which are outside 45 edges or arcs. See the IEEE Standard Dictionary of 
the polyhedron. Three-dimensional rendering is Electrical and Electronics Terms, Fourth Edition, 1988. 
thereby greatly simplified because the position of every This graph allows the renderer to quickly determine 
three-dimensional object need not be recalculated each what is on the other side of the hole in a polyhedron. If 
time the reference changes. there is a connection, then the hole is analogous to a 

One exception to the above rule concerning contain- 50 door leading into a second polyhedron. Since the 

ment occurs when a polyhedron includes a hole in the pointer is bidirectional, looking back from the second 

surface thereof. The hole in the polyhedron allows the polyhedron through the same hole allows the renderer 

contents of the polyhedron to be visible, but only within to see into the first polyhedron. FIG. 4 illustrates a plan 

the limited range of this hole. Thus, the image of an view of a plurality of three-dimensional rooms or rect- 

interior polyhedron needs to overlap the image of the 55 angular solids A, B, C, D and E, with a pyramidal solid 

hole in order for the interior polyhedron to be consid- contained in room E. The outside word is shown as W. 

ered when rendering from the outside. Thick lines indicate holes, with circular arrows indicat- 

According to the invention, holes may be defined ing bidirectionality. The reference or observer is shown 

between logically connected surfaces of two polyhedra. as 20. 

The hole provides visible access between the interiors 60 FIG. 5 illustrates the bidirectional graph which speci- 

of the polyhedra without having to conduct lengthy fies the connections among the polyhedra surfaces of 

search operations to determine if the polyhedron is FIG. 4. As shown in FIG. 5, polyhedra A and C are 

visible, and also to find other polyhedra which are not connected, polyhedra A and P are connected, polyhe- 

m fact visible but whose image overlaps the hole. Since dra C and D are connected, polyhedra C and E are 

the surfaces of the polyhedra are logically connected, 65 connected and polyhedra C and B are connected, 

the hole need only be denned in one surface and can be The connection graph structure of FIG. 5 may be 

"inherited" by the other connected surfaces. Thus, the enhanced by adding the holes which connect the van- 

issue of resolving facing holes is easily solved. ous connected surfaces. Such a connection graph struc- 
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ture with holes is shown in FIG. 6. Referring to FIG. 6, 
the connection graph structure with holes for the three- 
dimensional space of FIG. 4 is shown. As shown, poly- 
hedron A is connected to the outside world and to 
polyhedra C and D with holes. Polyhedron C is con- 
nected to polyhedron A with a hole and to polyhedra B 
and D with holes. Two holes are shown connecting 
polyhedra C and E. A single hole connects polyhedra C 
and B, and a single hole connects polyhedra C and D. 
Polyhedron D is also connected to the outside world by 
a hole. 

An example of the use of the connection graph of 
FIG. 6 to render the three-dimensional space of FIG. 4 
will now be described. A detailed object oriented pro- 
gramming implementation will then be described. 

Referring to FIG. 7, it is assumed that the observer is 
in the outside world W within which polyhedra A-E 
are contained. The position of the observer is shown in 
FIG. 7 at 20, using a circle to indicate position and a 
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product marketed under the trademark Virtus Walk- 
Through TM by Virtus Corporation, the assignee of the 
present invention. This program product was imple- 
mented in object oriented programming language. An 
overview of object oriented programming will first be 
described followed by a complete description of the 
object oriented programming implementation of the 
present invention. 

Object oriented programming is a computer pro- 
gramming technique which provides reusable and easily 
expandable programs. In contrast with known func- 
tional programming techniques, which are not easily 
adaptable to new functional requirements and new 
types of data, object oriented programs are reusable and 
expandable as new requirements arise. 

In an object oriented programming system, the pri- 
mary focus is on data, rather than functions. Object 
oriented programming systems are composed of a large 
number of "objects". An object is a data structure and a 



radial line through the circle to indicate the pointing 20 set of operations or functions that can access that data 



direction of the observer. 

As shown in FIG. 7, assume that the observer is out- 
side room A in the location shown. First, the room 
closest to the observer (A) is found and is rendered. 
During the rendering operation a hole 22 in the surface 25 
in front of the observer is found. The hole 22 is clipped 
to the current clipping area 21 which then becomes the 
clipping region. The clipping operation is a well known 
graphics operation which allows a graphics system to 
draw a line or polygon on the graphics screen, such that 30 
only the part of the object that is being drawn that 
overlaps the clipping object is actually drawn. Accord- 
ingly, the hole 22 in the front surface is clipped to the 
current clipping area 21 which then becomes the clip- 
ping region. See FIG. 8. The inside surfaces of room A 35 
are then drawn, clipped to the current clipping region. 

When each of these interior surfaces is drawn, a 
search is performed for holes in the surface using the 
connection graph structure with holes of FIG. 6. If 



structure. Each operation (function) that can access the 
data structure is called a "method". Each defined object 
will usually be manifested in a number of "instances". 
Each instance contains the particular data structure for 
a particular example of the object. 

Object oriented programming systems provide two 
primary characteristics which allow flexible and reus- 
able programs to be developed. These characteristics 
are referred to as "encapsulation" and "inheritance". 
The data of an object is encapsulated by its methods 
(functions). A wall of code has been placed around each 
piece of data. All access to the data is handled by the 
surrounding methods. Data independence is thereby 
provided because an object's data structure is accessed 
only by its methods. Only the associated methods know 
the internal data structure. This ensures data integrity. 

The "inheritance" property of object oriented pro- 
gramming systems allows previously written programs 
to be broadened by creating new superclasses and sub- 



there is a hole, a check is made to see if the hole con- 40 classes of objects. New objects are described by how 



nects to another polyhedron at that surface. If it does 
not, then the hole connects back to the outside world W 
and the exterior space is drawn from this position. The 
hole is then clipped to the current clipping region. If the 
result is a nil region, this indicates that the hole is not 45 
visible from the observer's point of view. 

After moving through all of the holes in the surface, 
a check is made to see if there are any connections. If 
there are, then a check is made to see if there are any 
holes in the surfaces to which the connections are 50 
linked, and these holes are dealt with as described 
above. The completed rendered surface is shown in 
FIG. 9. 

As described above, the use of a containment tree and 
a connection graph with holes allows a representation 55 
of the three-dimensional space to be efficiently rendered 
in real time. Based upon the description already pro- 
vided, this rendering system can be implemented on any 
computer graphics system by one having skill in the art, 
using conventional programming techniques or special 60 
purpose hardware. However, for purposes of describing 
the best mode contemplated by the inventors for carry- 
ing out the invention, an object oriented programming 
description will now be provided. 



Object Oriented Programming Overview 

As described above, the present invention was first 
implemented by the present inventors in a program 
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they differ from preexisting objects so that entirely new 
programs need not be written to handle new types of 
data for functions. Each subclass "inherits" the frame 
and methods of its superclass. 

In an object oriented system, a high level routine 
requests an object to perform one of its methods by 
sending the objects a ''message" telling the object what 
to do. The receiving object responds to the message by 
choosing the message that implements the method 
name, executing this method, and then returning con- 
trol to the calling high level routine along with the 
results of the method. A more detailed description of 
the design and operation of an object oriented program 
is provided in the textbook entitled "Object Oriented 
Software Construction" by Bert rand Meyer, published 
by Prentice Hall in 1988, the disclosure of which is 
incorporated herein by reference. 

Object Oriented Programming Implementation 

A detailed description of an object oriented imple- 
mentation of the three-dimensional rendering system of 
the present invention now be described. In the descrip- 
tion, "methods" "functions", "procedures" and "ac- 
tions" all refer to methods as defined above. As is well 
known to those having skill in the art, a description of 
the data and the methods that the object uses is suffi- 
cient to replicate the system. In the following descrip- 
tion, no assumptions are made about the particular pro- 
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gramming language for implementing the system except 
that it need be an object oriented language. Examples of 
appropriate object oriented languages are Smalltalk, 
Eiffel, C+ +, Borland Corporation's Turbo Pascal and 
Apple Computer Corporation's Object Pascal. 5 

The following definitions are used to describe the 
data. There are equivalent analogues in all object ori- 
ented programming languages if they are simple type, 
otherwise, they are defined in terms of the simple types, 
i.e. as collections of simple type information: 10 
boolean — a value that represents only TRUE or 

FALSE. It can have no other value associated with 

it. 

integer — a whole number where the range is deter- 
mined by the computer system running the program. 15 
Typical ranges are —32,000 to 32,000. 

double — a high precision natural number which has the 
ability to represent both very large numbers and frac- 
tional values. 

Point — a compound type number which includes a dou- 20 
ble representing the x-coordinate value and a double 
representing the y-coordinate value. 
Point3D — a compound type which is made up of three 
doubles representing the x, y and z coordinate values. 
*x— a star in front of a data type means that the object 25 
specified is not the actual data, but the address of this 
data which is likely someplace else in memory. This 
allows objects to refer to other objects without hav- 
ing to contain the entire contents of the other object 
That is, this object knows where that other object is 30 
in memory, so if it needs information from that other 
object, it knows where to find it 
Array — this is a compound type that is an ordered col- 
lection of the same type objects. This type can be of 
any length, defined as necessary. When an Array is 35 
specified, the type of the objects that make up the 
array are also specified. Thus, "integer Array x" 
represents a pointer to an Array of integers. 
The above definitions are used before the name of the 
data that is being described to define the type of that 40 
data. Thus "integer x" means that the data referred to 
by x is of the integer type. 

In the description which follows, methods are de- 
scribed by first specifying their return values, followed 
by the name of the method, followed by a list of argu- 45 
ments, or values that are passed to the method in paren- 
theses. The arguments are also specified as to their type. 
If there is no return value, or no arguments, the word 
"void" is used to specify this fact Examples of methods 
are: 50 
integer methodName(integer firstArg, integer Array 

,secondArg) 
void anotherMethod(void). 

A general description of the Objects will now be 
provided, followed by a detailed description of each 55 
object and its methods: 

Clip — The implementation of the Clip object type is 
well known to those having skill in the art of object 
oriented graphics programming. The preferred em- 
bodiment uses a polygonal clipping object, (not nec- 60 
essariiy rectangular) such as that described in section 
19.1 of the above mentioned Foley et al. textbook 
(pages 924-945), the disclosure of which is hereby 
incorporated herein by reference, or a Macintosh 
clipping region as described in the publication Inside 65 
Macintosh, published by Apple Computer Corpora- 
tion, pages I-135-I-162 and I-181-I-187, Volume 1, 
chapter 6, the disclosure of which is hereby incorpo- 
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rated herein by reference. The Clip object allows the 
system to draw a line or polygon on the computer 
display such that only the part of the object being 
drawn that overlaps the clipping object is actually 
drawn. 

Polyhedron— As already described, three-dimensional 
spaces are described in terms of polyhedra. The poly- 
hedron object type has methods that are used to con- 
struct the polyhedra, manipulate and transform them, 
render them, and ultimately to destroy them. It in- 
cludes containment information about associated 
polyhedra such as the polyhedron that contains this 
polyhedron and a list of all the polyhedra that this 
one contains, the data that defines the polyhedron 
and is used to actually create it, and the actual 3D 
polyhedron data such as the list of point3D and the 
surfaces that use these points, and that are used to 
render the polyhedron. 

Surface— This object type defines a surface of a polyhe- 
dron. It contains an array of indices into the polyhe- 
dron's list of points that define the outline of the 
surface. It also includes surface feature information. 
Surface features are holes in the surfaces. Thus a door 
or window is defined as a surface feature of a walL A 
Surface object can have any number of surface fea- 
tures. In addition to the surface feature, the surface 
includes the polyhedron connection information. A 
surface may be connected to any number of polyhe- 
dra. 

Feature — This object type is simply a list of point3D 
that define a hole in the surface of a polyhedron. 

Camera — This object type specifies both the position 
and the orientation in which to move the object. 
Typically, this represents the position and orientation 
of the camera (or observer). The standard method is 
to presume that the camera is fixed in space and at the 
center of the world. Thus, the objects are rotated 
around it by first translating them so that the camera 
is at the origin and then the objects are rotated around 
this origin. The resulting data is then rendered. The 
representation of the data is a matrix representation 
described in Chapter 5.6 pages 213-217 of the Foley, 
et al textbook. 

A detailed description of the objects, and their data 
and methods, will now be provided. 

Clip 

Since the methods of clipping are well known to 
those skilled in the graphics art, a detailed implementa- 
tion need not be described. Two unique methods used 
by the present invention will be described. 

Methods 

DrawClip(Point Array *polygon) 

This method draws the polygon on the screen only 
inside the clipping region. It draws the intersection of 
the polygon and the dipping region. Note that the re- 
gion itself is (usually) a polygon. 

dip * MakeClip(Point Array ^polygon) 

This method creates a new clipping region that is the 
intersection of the old clipping region and the polygon 
that is passed into it. 
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This object contains the 4 by 4 matrix that is de- 
scribed in Chapter 5.6 pages 213-217 of the Foley, et al. 
textbook and a single method: 5 

Point Transform(Point3D fromPoint) 

This takes a 3-D point and transforms it using the 
4X4 matrix into the 2-D screen position point 



Polyhedron 



10 



The observer or reference is always inside a polyhe- 
dron. The polyhedron may in fact be the global, or 
world polyhedron which has no bounds, but the ob- 
server is inside it The rendering process first deter- 15 
mines which polyhedron the reference is in. The cam- 
era's spatial position (x,y,z coordinates) is used to find 
the current polyhedron, and then call that polyhedron's 
Draw3D method described below. In calling the 
Draw3D method, the following arguments, are pro- 20 
vided: the ClipRegion set to the entire screen (or win- 
dow port), the current camera pose, and inside =- 
TRUE, because we are in fact inside the polyhedron we 
are beginning to render. Note that the world polyhe- 
dron has no surfaces. Thus in the world polyhedron, 25 
only the contents of this polyhedron are drawn. 

Data 

Polyhedron *container 

This is the address of the polyhedron that contains 
this polyhedron. Containment is very specific; the con- 
tainer polyhedron actually spatially encloses this poly- 
hedron. There is an ultimate container of polyhedra and 
this is referred to as the world. It has a value of 0 for the 
container. 



Polyhedron Array*contents 

Just as this polyhedron is contained within another 
polyhedron, this is the address of an array that contains ^ 
pointers to all of the polyhedra that are contained 
within this one. 

Point3D Array points 

This is the array of the actual vertex points of the 45 
polyhedron. 

Point Array posedPoints 

This is the array of the vertex points of the polyhe- 
dron after the camera transformation has been applied. 
Note that the result is 2-D and is mapped to the screen 
or window port, using known techniques. 



Surface Array surfaces 

These surface objects contain an ordered list of indi- 
ces into the point array above. An ordered list is used 
instead of the actual point value because a point is typi- 
cally shared by more than one surface. If this point 
needs to be spatially transformed; it need only be trans- 
formed once, and each surface has access to the new 
point Otherwise, it would need to be transformed mul- 
tiple times for each surface. 

Methods 

Polyhedron*InsidePolyhedron(Point3D position) 

This method returns the smallest polyhedron that the 
position point is inside. It comprises the following steps: 
• Determine if the position is inside this polyhedron. 



• If not, return NIL. 

• If inside, for each contained polyhedron call it's 
InsidePolyhedron method until a non-NIL value is 
returned. 

• If non-NIL is returned, return that value. 

• Else, return a pointer to THIS polyhedron. This 
pointer means that we are inside of this polyhe- 
dron, but outside of the contents of this polyhe- 
dron. 

• Draw3D(Oip *clipRegion,Camera *newPose, 
boolean inside) This method comprises the follow- 
ing steps: 

• Call the Calc3D method below to reorient this 
polyhedron to the observer's position. 

• Call the surface's Calc3D method for each surface 
which reorients each surface's surface vector. 

• Call the PrawSurface method for each surface to 
draw that surface, including the surface features. 

• Sort the contents of the polyhedron by distance. 

• Call Draw3D for each of the content polyhedra, 
starting with the furthest, until the closest is ob- 
tained. 

Calc3D(Camera * new Pose) 

This method is used to reorient the object to the 
camera's orientation. For each point, transform the 
point by the method described in Chapter 5.6 pages 
213-217 of the Foley, et al. textbook. Place the resulting 
point into the corresponding position in the posedPoints 
array. 

Surface 
Data 

35 Polyhedron *owner 

This is a pointer to the owning polyhedron. This 
gives the surface object access to the polyhedron's 
posedPoints array and to the container polyhedron. 

integer Array index 

This is a list of indices that apply both to the points 
array and the posedPoints array. It is the ordered indi- 
ces of the vertices of this surface. Note that this list is 
always defined such that the direction is clockwise from 
the outside. 

Point3D normal 

This is the surface normal to the surface. It is a point 
defined to be a unit distance from the origin which 
represents the vector that is perpendicular to each point 
on the surface. It's length is unimportant, as long as it is 
non-zero and oriented in the outside direction (out of 
the polyhedron). 



30 



55 Feature Array surfaceFeatures 

This is an array of surface features, which are defined 
to be ordered lists of 3-D points that represent holes in 
the surface object. 

60 Polyhedron Array connected 

Surface Array connectedSurface 

This is an array of all of the polyhedra that are con- 
nected to this surface and the surfaces to which they are 
65 connected. This is analogous to two rooms being next to 
each other and sharing a wall. This connection is bidi- 
rectional. Thus, if a polyhedron is listed in this surface's 
connected array, then that polyhedron's surface has this 
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polyhedron listed in its connection array. This allows method using the indexed array of points as the 

two polyhedra to share surface features such as doors. argument. 

, , • For each surface feature in the surface: 

• Transform this surface feature (which is an array 

CaJc3D (Camera *newPose) 5 of 3-D points) using the camera object 

Applies the new camera orientation to the surface * a r new cli P "^S MakeClip and the 

normal vector. If we assume that the camera is posi- ?f[ ace fea * re P° lnt **** argument 

tioned along the z-asis, this results in a positive or nega- • » Draw * D ? f ™ 

tive value for the z component, depending on which ln polyhedron^ We are now drawing the made of 

side of the surface we are on. If we are on the same side, 10 ^ with Ae new calculated chp 

this value is positive. If we are on the opposite side, this reglon ' camm P 051 * 10 " 811(1 ««>de=TRUE. 

value is negative. Moving Through Three-Dimensional 

DrawSurface<Clip »region, boolean inside) Space— Overview 

™„ ;« „ *„ o r> -n™.:„„ ,„„:„„ 15 The method and apparatus for moving the reference 

™ P . • *T • I ? ? P ^ g • ^ through the simulated three-dimensional space accord- 

We only draw made , A* region. Boolean .ns.de ,s a flag » E2eito ^g 

Oat indicates whether we are drawing this polygon a ibn ^ aaalmil space> k b desirable J 

rrom tneinsweornot. . __ TT _, allow the reference or observer to move with six de- 

. If we are drawing from the inside (made is TRUE) 2Q of freedom ^ ^curziely represents the 

and the normal^ vector is positive, then call Dra- ^ degrecs of avai|ab , e to a ^ observer 

wmtenor(Chp region) moving through a three-dimensional space. Unfortu- 

• cm 6 T ? raWmg thB ° Utade W nately, * has heretofore been difficult to allow a user to 
FALSE) and the normal vector .s negative, then move ^ h real time tnreeKlimens i ona] computer 

caU DrnwExtenorCQip *reg.on) 25 generated scenes using typical computer-based data 

. Else this surface s not visible and we need not draw mput devices . Known computer input devices include a 

xt ' keyboard and a number of pointing devices such as a 

Drawinterior(Clip *region) joystick, trackball or mouse. 

_ , „ , , . . The keyboard does not provide a natural technique 

• Draw the surface through the clipping region by ^ for mQy . throu ^ a thr ^ mei ri onal s because 

making the call to the clipping region s DrawChp it requires muIti le keystrokes to move, or it assumes a 

method using the indexed array of points as the velochy set by numeric keys and the other keys are used 

argument. to change direction. Joysticks are easier to use than a 

• For each surface feature in the surface: keyboard, but are useful primarily for airplane simula- 

• Transform this surface feature (which is an array 35 tors where six degrees c f freedom is typically not neces- 
of 3-D points) using the camera object sary. The joystick typically assumes a forward velocity 

• Create a new clip object using MakeClip and the ^ ^ user ^ Ae j oystick to modify the direction, 
surface feature point list as the argument. If the There have been a number of mouse control systems 
resulting clip object is NIL, go to next surface used with personal computer flight simulators but these 
feature. 40 have been based upon the control system used by the 

• If there are any connected polyhedra to this sur- joystick. Unfortunately, if the user is moving through a 
face, call each of the connected polyhedrons space, such as a building, and may even be modifying 
Draw3D method with the new calculated clip and creating the space, the inability to stop, move back- 
region, the camera position and inside = TRUE. wards, and in fact move in six degrees of freedom is a 

• If there are no connected polyhedra to this sur- 45 major hinderance. 

face, find the containing polyhedron of the One known technique which allows the user to stop 

owner polyhedron and make a call to the and move backwards through a three-dimensional space 

Draw3D method with the new calculated chp i s illustrated by the above described computer adven- 

region, the camera position and inside— TRUE, ture game 'The Colony". Unfortunately, this technique 

Note that we are still INSIDE the containing 50 does not allow the user to easily move directly to the 

polyhedron. left or right, to change altitude or to change pitch or 

• For each connected polyhedron that is connected roll. In other words, motion along six degrees of free- 
to this surface, determine the surface of the polyhe- dom is not provided. 

dron that is connected to this one. Then, for each According to the invention, the pointing device is 

surface feature in the surface: 55 used in combination with first and second function se- 

• Transform this surface feature (which is an array lection means to provide six degrees of freedom 
of 3-D points) using the camera object. through the three-dimensional space rendition. Refer- 

• Create a new clip object using MakeClip and the ring again to FIG. 1, the function selection means are 
surface feature point list as the argument preferably a pair of keys 16a, 16b on mouse 15, which 

• Call the connected polyhedrons Draw3D 60 may be selected or held down while the mouse is moved 
method with the new calculated clip region, the along the X or Y axis to provide six degrecs of freedom, 
camera position and inside =TRUE. Alternatively, a pair of keys on the keyboard 14 may be 

This allows two polyhedra to share the same hole used. The keys may be used in a toggle mode in which 

when they are connected. the function is implemented until the key is depressed 

_ — > ± . . 65 again, or in a hold down mode in which the key must be 

Draw Extenor(Clip region) hdd down while ^ mouse k moved t0 fanp^n^ the 

• Draw the surface through the clipping region by function. Other function selection keys are known to 
making the call to the clipping region's DrawClip those having skill in the art. 
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In general, the present invention provides movement 
in a first plane in a three-dimensional space rendition in 
response to movement of the mouse. It also provides 
movement in a second plane in the three-dimensional 
space rendition in response to selection of the first key 5 
and movement of the mouse. Selection of the second 
key provides rotation in the three-dimensional space in 
conjunction with movement of the mouse. 

A preferred embodiment of the present invention will 
now be described in which the combination of mouse 10 
movement and function selection keys have been opti- 
mized to provide user friendly movement through a 
three-dimensional space along six degrees of freedom. 
Referring now to FIG. 10, the action of the mouse 
when neither of the function keys are selected is shown. 15 
As shown, placement of the mouse on the vertical axis 
moves the reference forward or backward in the three- 
dimensional space. Movement to the left and right turns 
the user to the left and right in the three-dimensional 
space. In other words, yaw is changed to allow left 20 
rotation and right rotation. 

Referring now to FIG. 11, selection of the first func- 
tion selection key, here the "option" (OP) key, and 
movement of the mouse in the Y (vertical) direction 
causes the reference to rise up or sink down, Le. to 25 
move along the Z axis of the three-dimensional space. 
Similarly, movement of the mouse in the horizontal 
direction while pressing the option key allows the refer- 
ence point to move sideways in the three-dimensional 
space. 30 

Referring now to FIG. 12, selection of the second 
function key such as the "command" key (the symbol 
for which this is shown in FIG. 12), in combination with 
movement of the mouse in the vertical (Y) direction, 
allows the reference to look up or look down, i.e. to 35 
change pitch. Movement of the mouse in the horizontal 
direction (X axis) in combination with selection of the 
command key, allows the reference to roll to the left or 
roll to the right 

The above described combination of function, selec- 40 
tion and mouse movement allows the user to easily 
move through a three-dimensional space. 

Movement Through Three-Dimensional 

Space-Detailed Description 45 

The detailed operational steps in moving the user 
through three-dimensional space in response to actua- 
tion of the function keys and movement of the mouse 
will now be described. These operational steps may be 
performed during the method of the Camera object 50 
which was described in detail above. 

FIGS. 13A and 13B illustrate the coordinate system 
used for the three-dimensional space, and a representa- 
tion of the grid and cursor on a two-dimensional graph- 
ics display of a three-dimensional space. As shown in 55 
FIG. 13A, p is the position point and consists of Px, Py 
and p,. As also shown in FIG. 13A, v is the orientation 
vector and consists of v w /i, v pit ^ and Vjm*. The calcula- 
tions described below are preferably performed on inte- 
gers with integer results. Division rounds towards zero 60 
so that 5/3 = 1 and — 5/3= — 1. Finally, in the calcula- 
tions described below, x**y means All of the calcula- 
tions described below assume that a horizontal control 
reference h^and a vertical control reference v^are 
known. The horizontal control reference is defined as 65 
the maximum horizontal position divided by six 
(hmax/6), and the vertical control reference is defined as 
the maximum vertical position divided by seven 
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(vmox/7), because there are 6 horizontal grid lines and 7 
vertical grid lines from the center. In other grid spac- 
mgs, other constants may be substituted. 

Referring to FIGS. 14-19, the detailed operational 
steps for implementing the six degrees of movement 
according to the present invention will now be de- 
scribed. The sequences of operational steps are all simi- 
lar. In particular, first a control value is read and a 
temporary value is calculated. The temporary value is 
tested, and a velocity value is calculated depending on 
whether the temporary value is less than zero, equal to 
zero, or greater than zero. Once the velocity value has 
been calculated, it is added to the existing value. Since 
all of the calculations involve the same overall struc- 
ture, they will not be described individually. 

Referring now to FIG. 14, the steps for moving the 
observer forward or backward through the three-di- 
mensional space are described. These steps are per- 
formed when neither of the selection keys 16a, 166 
(FIG. 1) are selected and the mouse is moved forward 
or backward. FIGS. 20A-20J illustrate the result of the 
steps of FIG. 14 on a three-dimensional space represen- 
tation. FIGS. 21A-21J illustrate moving forward and 
backward when the mouse is placed at a faster velocity 
than shown in FIGS. 20A-20J. FIGS. 22A-22E illus- 
trate moving backwards. 

Referring to FIG. 15, the steps for turning left or 
right (yaw) are shown. These steps are performed in 
response to neither of the function keys 16a, 166 being 
selected, and the mouse being moved to the left or to 
the right. FIGS. 24A-24E illustrate the graphical re- 
sults of turning to the left and FIGS. 25A-25E illustrate 
the graphical results of turning to the right in response 
to the steps of FIG. 15. FIGS. 23A-23E illustrate the 
combined motion of moving forward and turning right 
in response to the steps of FIGS. 14 and 15 which are 
performed upon forward and rightward movement of 
the mouse without selecting one of the selection keys 
16a, 166. 

FIG. 16 illustrates the steps for moving up or down in 
a three-dimensional representation. These steps are per- 
formed upon selection of the first selection key 16a and 
movement of the mouse forward or backwards. FIGS. 
26A-26E and 27A-27E illustrate the graphical results 
of moving up and moving down, respectively, in re- 
sponse to the steps of FIG. 16. 

FIG. 17 illustrates the steps performed to shift left or 
shift right, in response to selection of the first selection 
key 16a and movement of the mouse to the left or the 
right. FIGS. 28A-28E and 29A-29E illustrate the re- 
sults of these steps for moving left and moving right, 
respectively, in the graphical representation of the 
three-dimensional space. 

FIG. 18 illustrates the steps for increasing or decreas- 
ing pitch in response to selection of the second function 
selection key 166 and moving the mouse forward and 
backward. The results of these steps on the graphical 
representation of the three-dimensional space are shown 
in FIGS. 30A-30E and 31A-31E for increasing and 
decreasing pitch, respectively. 

Finally, FIG. 19 illustrates the steps in rolling to the 
left or right in response to selection of the second func- 
tion selection key and movement of the mouse to the 
left or right. FIGS. 32A-32E and 33A-33E illustrate 
the results of these steps on a graphics representation of 
the three-dimensional space for rolling to the left or 
rolling to the right respectively. 
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It will be understood by those having skill in the art 
that the individual steps for performing the individual 
functions to move along six degrees of freedom are 
known. However, it has heretofore not been known to 
use these steps in combination with movement of a 5 
mouse and selection of a pair of selecting means, in the 
particular relationship described, to effect user-intuitive 
movement through a two-dimensional representation of 
a three-dimensional space with all six degrees of free- 
dom. 10 

In the drawings and specification, there have been 
disclosed typical preferred embodiments of the inven- 
tion and, although specific terms are employed, they are 
used in a generic and descriptive sense only and not for 
purposes of limitation, the scope of the invention being 15 
set forth in the following claims. 

That which is claimed: 

1. A graphics display system comprising: 
two-dimensional graphics displaying means; and 
graphics processing means for a plurality of three-di- 20 

mensional objects, a first one of said objects being 
contained within a second one of said objects, said 
graphics processing means comprising: 
means for representing said plurality of three-di- 
mensional objects as a corresponding plurality of 25 
convex polyhedra; 
means for. indicating containment relationships 
among said plurality of convex polyhedra corre- 
sponding to containment relationships among 
said plurality of three-dimensional objects, in- 30 
eluding said first object which is contained 
within said second object; and 
means for rendering a two-dimensional representa- 
tion of the plurality of three-dimensional objects 
by selectively rendering only some of said con- 35 
vex polyhedra on said graphics displaying 
means, based upon the containment relationships 
in said containment indicating means, such that 
said first object is not rendered on said graphics 
displaying means upon rendering said second 40 
object from a perspective outside said second 
object, and said second object is not rendered on 
said graphics displaying means upon rendering 
said first object from a perspective inside said 
first object. 45 

2. The graphics display system of claim 1 wherein 
said containment indicating means comprises a contain- 
ment logical tree for indicating said containment rela- 
tionships. 

3. The graphics display system of claim 1 further 50 
comprising means for indicating connection relation- 
ships among said plurality of three-dimensional objects. 

4. The graphics display system of claim 1 wherein 
said connection indicating means comprises means for 
indicating connection of at least a first surface of a third 55 
one of said convex polyhedra to at least a second sur- 
face of a fourth one of said convex polyhedra. 

5. The graphics display system of claim 4 wherein 
said connection indicating means further comprises 
means for indicating a location of at least one hole be- 60 
tween said at least a first surface which is connected to 
said at least a second surface. 

6. The graphics display system of claim 3 wherein 
said connection indicating means comprises a connec- 
tion logical graph for indicating said connection rela- 65 
tionships. 

7. A graphics display method for displaying a plural- 
ity of three-dimensional objects, a first one of said ob- 
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jects being contained within a second one of said ob- 
jects, said graphics display method comprising the steps 
of: 

representing in a graphics processor, said plurality of 
three-dimensional objects as a corresponding plu- 
rality of convex polyhedra; 

generating in the graphics processor, a representation 
of containment relationships among said plurality 
of convex polyhedra corresponding to contain- 
ment relationships among said plurality of three-di- 
mensional objects including said a first object 
which is contained within said second object; and 

displaying the plurality of convex polyhedra on a 
two-dimensional display by selectively rendering 
only some of said polyhedra on said two-dimen- 
sional display based on the containment relation- 
ships represented in the graphics processor, such 
that said first object is not rendered on said graph- 
ics display upon rendering said second object from 
a perspective outside said second object, and said 
second object is not rendered on said graphics 
display upon rendering said first object from a 
perspective inside said first object 

8. The graphics display method of claim 7 wherein 
said generating step comprises the step of generating a 
containment logical tree for indicating the containment 
relationships among said plurality of convex polyhedra. 

9. The graphics display method of claim 7 further 
comprising the step of generating a representation of 
the connection relationships among said plurality of 
convex polyhedra corresponding to connection rela- 
tionships among said plurality of three-dimensional 
objects. 

10. The graphics display method of claim 9 wherein 
said connection relationships generating step comprises 
the step of representing that at least a first surface of a 
third one of said convex polyhedra is connected to at 
least a second surface of a fourth one of said convex 
polyhedra. 

11. The graphics display method of claim 10 wherein 
said surface connection representing step further com- 
prises the step of representing a location of at least one 
hole between said at least a first surface which is con- 
nected to said at least second surface. 

12. The graphics display method of claim 9 wherein 
said connection relationships generating step comprises 
the step of generating a connection logical graph for 
representing said connection relationships. 

13. Apparatus for moving through a computer- 
generated three-dimensional space rendition, said mov- 
ing apparatus comprising: 

two-dimensional graphical display means, for dis- 
playing a three-dimensional space rendition 
thereon; 

two-dimensional pointing means, for moving a refer- 
ence on said two-dimensional graphical display 
means in response to two-dimensional movement 
of said two-dimensional pointing means; 

function selection means; 

first means for moving in a first plane in the displayed 
three-dimensional space rendition, in response to 
movement of said two-dimensional pointing means; 
and 

second means for moving in a second plane in the 
displayed three-dimensional space rendition, in 
response to the combination of selection of said 
function selection means and two-dimensional 
movement of said two-dimensional pointing means; 
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wherein said second moving means comprises means 
for rising up and sinking down in a vertical plane of 
the displayed three-dimensional space rendition, 
and for moving left and moving right in the hori- 
zontal plane of the displayed three-dimensional 
space rendition, in response to the combination of 
selection of said first function selection means and 
forward, backward, left and right movement, re- 
spectively, of said two-dimensional pointing 
means. 

14. The moving apparatus of claim 13 wherein said 
first moving means comprises means for moving for- 
ward, moving backward, turning left and turning right 
in a horizontal plane of the displayed three-dimensional 
space rendition, in response to forward, backward, left 
and right movement, respectively, of said two-dimen- 
sional pointing means. 

15. The moving apparatus of claim 13 wherein said 
two-dimensional pointing means comprises a mouse, 
and wherein said function selection means comprises a 
key. 

16. The moving apparatus of claim 15 wherein said 
key comprises a key on said mouse. 

17. The moving apparatus of claim 15 further com- 
prising a keyboard including a plurality of keys, and 
wherein said key comprises a predetermined one of said 
keyboard keys. 

18. Apparatus for moving through a computer- 
generated three-dimensional space rendition, said mov-. 
ing apparatus comprising: 

two-dimensional graphical display means, for dis- 
playing a three-dimensional space rendition 
thereon; 

two-dimensional pointing means, for moving a refer- 
ence on said two-dimensional graphical display 
means in response to two-dimensional movement 
of said two-dimensional pointing means; 

function selection means; 

first means for moving in a first plane in the displayed 40 
three-dimensional space rendition, in response to 
movement of said two-dimensional pointing means; 
and 

second means for moving in a second plane in the 
displayed three-dimensional space rendition, in $5 
response to the combination of selection of said 
function selection means and two-dimensional 
movement of said two-dimensiorial pointing means; 

second function selection means; and 
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two-dimensional graphical display means, for dis- 
playing a three-dimensional space rendition 
thereon; 

two-dimensional pointing means, for moving a refer- 
ence on said two-dimensional graphical display 
means in response to two-dimensional movement 
of said two-dimensional pointing means; 

function selection means; and 

means for moving in six degrees of freedom through 
the displayed three-dimensional space rendition, 
said six degrees of freedom including translation 
along three orthogonal axes and rotation about said 
three orthogonal axes, in response to the combina- 
tion of selection of said function selection means 
and two-dimensional movement of said two-dimen- 
sional pointing means; 

wherein said function selection means comprises first 
and second function selection means; 

wherein said six degrees of freedom comprise transla- 
tion along mutually orthogonal x, y and z axes, and 
rotation about said x, y and z axes, and wherein said 
two-dimensional pointing means moves the refer- 
ence horizontally and vertically along said graphi- 
cal display means, and wherein said moving means 
comprises: 

means for translating through the displayed three-di- 
mensional space rendition along said y axis in re- 
sponse to vertical movement of said two-dimen- 
sional pointing means and for rotating about said z 
axis in response to horizontal movement of said 
two dimensional pointing means; 
means for translating through the displayed three-di- 
mensional space rendition along said z axis in re- 
sponse to vertical movement of said two dimen- 
sional pointing means and selection of said first 
function selection means, and for translating along 
said x axis in response to horizontal movement of 
said two-dimensional pointing means and selection 
of said first function selection means; and 
means for rotating in the displayed three-dimensional 
space rendition about said x axis in response to 
vertical movement of said two-dimensional point- 
ing means and selection of said second function 
selection means, and for rotating about said y axis 
in response to horizontal movement of said two-di- 
mensional pointing means and selection of said 
second function selection means. 
20. The moving apparatus of claim 19 wherein said 
two-dimensional pointing means comprises a mouse, 



means for rotating in die displayed^three-dimensional^ 50 and wherein said function selection means comprises a 

n 

21. The moving apparatus of claim 20 wherein said 
key comprises a key on said mouse. 

22. The moving apparatus of claim 20 further com- 
prising a keyboard including a plurality of keys, and 
wherein said key comprises a predetermined one of said 
keyboard keys. 

23. A graphics display system for rendering a three- 
dimensional space including a plurality of three-dimen- 
sional objects, a first one of said objects being contained 
within a second one of said objects, and for moving 
through the three-dimensional space rendition, said 
graphics display system comprising: 

two-dimensional graphics displaying means; 
two-dimensional pointing means, for moving a refer- 
ence on said two-dimensional graphics displaying 
means; 

function selection means; and 



space rendition, in response to the combination of 
selection of said second function selection means 
and two-dimensional movement of said two-dimen- 
sional pointing means; 
wherein said rotating means comprises means for 55 
rotating up and rotating down relative to a first axis 
in the horizontal plane of the displayed three-di- 
mensional space rendition, and for rotating up and 
rotating down relative to a second axis in the hori- 
zontal plane of the displayed three-dimensional 60 
space rendition, in response to the combination of 
selection of said second function selection means 
and forward, backward, left and right movement, 
respectively, of said two-dimensional pointing 
means. 65 
19. Apparatus for moving through a computer- 
generated three-dimensional space rendition, said mov- 
ing apparatus comprising: 
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graphics processing means, said graphics processing 
means comprising: 

means for representing said plurality of three-dimen- 
sional objects as corresponding plurality of convex 
polyhedra; 5 

means for indicating containment relationships 
among said plurality of convex polyhedra corre- 
sponding to containment relationships among said 
plurality of three-dimensional objects including 
said first object which is contained within said 1° 
second object; 

means for rendering a two-dimensional representa- 
tion of the plurality of three-dimensional objects by 
selectively rendering only some of said convex 
polyhedra on said graphics displaying means, based 15 
upon the containment relationships in said contain- 
ment indicating means, such that said first object is 
not rendered on said graphics displaying means 
upon rendering said second object from a perspec- 
tive outside said second object, and said second 2 ^ 
object is not rendered on said graphics displaying 
means upon rendering said first object from a per- 
spective inside said first object; and 

means for moving in six degrees of freedom through 25 
the displayed three-dimensional space rendition on 
said graphics displaying means, said six degrees of 
freedom including translation along three orthogo- 
nal axis and rotation about said three orthogonal 
axis, in response to the combination of selection of 30 
said function selection means and two-dimensional 
movement of said two-dimensional pointing means. 

24. The graphics display system of claim 23 wherein 
said containment indicating means comprises a contain- 
ment logical tree for indicating said containment rela- 35 
tionships. 

25. The graphics display system of claim 23 wherein 
said graphics processing means further comprises means 
for indicating connection relationships among said plu- 
rality of convex polyhedra. 40 

26. The graphics display system of claim 25 wherein 
said connection indicating means comprises means for 
indicating connection of at least a first surface of a third 
one of said convex polyhedra to at least a second sur- 
face of a fourth one of said convex polyhedra. 45 

27. The graphics display system of claim 26 wherein 
said connection indicating means further comprises 
means for indicating a location of at least one hole be- 
tween said at least a first surface which is connected to 
said at least a second surface. 50 

28. The graphics display system of claim 25 wherein 
said connection indicating means comprises a connec- 
tion logical graph for indicating said connection rela- 
tionships. 

29. The graphics display system of claim 23 wherein 55 
said function selection means comprises first and second 
function selection means. 

30. The graphics display system of claim 29 wherein 
said six degrees of freedom comprise translation along 
mutually orthogonal x,y and z axes, and rotation about 60 
said x, y and z axes, and wherein said two-dimensional 
pointing means moves the reference horizontally and 
vertically along said graphical displaying means, and 
wherein said moving means comprises: 

means for translating through the displayed three-di- 65 
mensional space rendition along said y axis in re- 
sponse to vertical movement of said two-dimen- 
sional pointing means and for rotating about said z 



axis in response to horizontal movement of said 
two dimensional pointing means; 

means for translating through the displayed three-di- 
mensional space rendition along said z axis in re- 
sponse to vertical movement of said two dimen- 
sional pointing means and selection of said first 
function selection means, and for translating along 
said x axis in response to horizontal movement of 
said two-dimensional pointing means and selection 
of said first function selection means; and 

means for rotating in the displayed three-dimensional 
space rendition about said x axis in response to 
vertical movement of said two-dimensional point- 
ing means and selection of said second function 
selection means, and for rotating about said y axis 
in response to horizontal movement of said two-di- 
mensional pointing means and selection of said 
second function selection means. 

31. The graphics display system of claim 23 wherein 
said two-dimensional pointing means comprises a 
mouse, and wherein said function selection means com- 
prises a key. 

32. The graphics display system of claim 31 wherein 
said key comprises a key on said mouse. 

33. The graphics display system of claim 31 further 
comprising a keyboard including a plurality of keys, and 
wherein said key comprises a predetermined one of said 
keyboard keys. 

34. A system for rendering a plurality of three-dimen- 
sional objects, a first one of said objects being contained 
within a second one of said objects, said rendering sys- 
tem comprising: 

two-dimensional graphics displaying means; 

means for indicating containment relationships 
among said plurality of three-dimensional objects 
including that said first object is contained within 
said second object; and 

means for displaying a two-dimensional representa- 
tion of said plurality of objects on said two-dimen- 
sional graphics displaying means, from a display 
perspective which is outside said second object, by 
rendering said second object and by not rendering 
said first object, based upon the containment rela- 
tionships in said containment indicating means, and 
for displaying a two-dimensional representation of 
said plurality of objects on said two-dimensional 
graphics displaying means, from a display perspec- 
tive which is inside said first object, by rendering 
said first object and by not rendering said second 
object, based upon the containment relationships in 
said containment indicating means. 

35. The rendering system of claim 34 wherein said 
containment indicating means comprises a containment 
logical tree for indicating said containment relation- 
ships. 

36. The rendering system of claim 34 further compris- 
ing means for indicating connection relationships 
among said plurality of three-dimensional objects. 

37. The rendering system of claim 36 wherein said 
connection indicating means comprises means for indi- 
cating connection of at least a first surface of a third one 
of said convex polyhedra to at least a second surface of 
a fourth one of said convex polyhedra. 

38. The rendering system of claim 36 wherein said 
connection indicating means comprises a connection 
logical graph for indicating said connection relation- 
ships. 
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39. The rendering system of claim 37 wherein said 
connection indicating means further comprises means 
for indicating a location of at least one hole between 
said at least a first surface which is connected to said at ^ 
least a second surface. 

40. A method for rendering a plurality of three-di- 
mensional objects, a first one of said objects being con- 
tained within a second one of said objects, said render- 
ing method comprising the steps of: 10 

indicating in a graphics processor, containment rela- 
tionships among said plurality of three-dimensional 
objects including that said first object is contained 
within said second object; 15 

displaying a two-dimensional representation of said 
plurality of objects on a two-dimensional graphics 
display, from a display perspective which is outside 
said second object, by rendering said second object 
and by not rendering said first object, based upon 20 
the containment relationships in the graphics pro- 
cessor, and 

displaying a two-dimensional representation of said 
plurality of objects in said two-dimensional graph- 2 5 
ics display, from a display perspective which is 
inside said first object, by rendering said first object 
and by not rendering said second object, based 
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upon the containment relationships in the graphics 
processor. 

41. The rendering method of claim 40 wherein said 
indicating step comprises the step of generating a con- 
tainment logical tree in the graphics processor, for indi- 
cating the containment relationships among said plural- 
ity of three-dimensional objects. 

42. The rendering method of claim 40 further com- 
prising the step of generating a representation of the 
connection relationships among said plurality of three- 
dimensional objects. 

43. The rendering method of claim 42 wherein said 
connection relationships generating step comprises the 
step of representing that at least a first surface of a third 
one of said three-dimensional objects is connected to at 
least a second surface of a fourth one of said three-di- 
mensional objects. 

44. The rendering method of claim 42 wherein said 
connection relationships generating step comprises the 
step of generating a connection logical graph in the 
graphics processor for representing said connection 
relationships. 

45. The rendering method of claim 43 wherein said 
surface connection representing step further comprises 
the step of representing a location of at least one hole 
between the at least a first surface which is connected to 
said at least a second surface. 

* * * * * 
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